Ensemble of deep learning language models to support the creation of living systematic reviews for the COVID-19 literature.

BACKGROUND: The COVID-19 pandemic has led to an unprecedented amount of scientific publications, growing at a pace never seen before. Multiple living systematic reviews have been developed to assist professionals with up-to-date and trustworthy health information, but it is increasingly challenging for systematic reviewers to keep up with the evidence in electronic databases. We aimed to investigate deep learning-based machine learning algorithms to classify COVID-19-related publications to help scale up the epidemiological curation process. METHODS: In this retrospective study, five different pre-trained deep learning-based language models were fine-tuned on a dataset of 6365 publications manually classified into two classes, three subclasses, and 22 sub-subclasses relevant for epidemiological triage purposes. In a k-fold cross-validation setting, each standalone model was assessed on a classification task and compared against an ensemble, which takes the standalone model predictions as input and uses different strategies to infer the optimal article class. A ranking task was also considered, in which the model outputs a ranked list of sub-subclasses associated with the article. RESULTS: The ensemble model significantly outperformed the standalone classifiers, achieving a F1-score of 89.2 at the class level of the classification task. The difference between the standalone and ensemble models increases at the sub-subclass level, where the ensemble reaches a micro F1-score of 70% against 67% for the best-performing standalone model. For the ranking task, the ensemble obtained the highest recall@3, with a performance of 89%. Using an unanimity voting rule, the ensemble can provide predictions with higher confidence on a subset of the data, achieving detection of original papers with a F1-score up to 97% on a subset of 80% of the collection instead of 93% on the whole dataset. CONCLUSION: This study shows the potential of using deep learning language models to perform triage of COVID-19 references efficiently and support epidemiological curation and review. The ensemble consistently and significantly outperforms any standalone model. Fine-tuning the voting strategy thresholds is an interesting alternative to annotate a subset with higher predictive confidence.

The Impact of the COVID-19 Pandemic and Associated Control Measures on the Mental Health of the General Population : A Systematic Review and Dose-Response Meta-analysis.

BACKGROUND: To what extent the COVID-19 pandemic and its containment measures influenced mental health in the general population is still unclear. PURPOSE: To assess the trajectory of mental health symptoms during the first year of the pandemic and examine dose-response relations with characteristics of the pandemic and its containment. DATA SOURCES: Relevant articles were identified from the living evidence database of the COVID-19 Open Access Project, which indexes COVID-19-related publications from MEDLINE via PubMed, Embase via Ovid, and PsycInfo. Preprint publications were not considered. STUDY SELECTION: Longitudinal studies that reported data on the general population's mental health using validated scales and that were published before 31 March 2021 were eligible. DATA EXTRACTION: An international crowd of 109 trained reviewers screened references and extracted study characteristics, participant characteristics, and symptom scores at each timepoint. Data were also included for the following country-specific variables: days since the first case of SARS-CoV-2 infection, the stringency of governmental containment measures, and the cumulative numbers of cases and deaths. DATA SYNTHESIS: In a total of 43 studies (331 628 participants), changes in symptoms of psychological distress, sleep disturbances, and mental well-being varied substantially across studies. On average, depression and anxiety symptoms worsened in the first 2 months of the pandemic (standardized mean difference at 60 days, -0.39 [95% credible interval, -0.76 to -0.03]); thereafter, the trajectories were heterogeneous. There was a linear association of worsening depression and anxiety with increasing numbers of reported cases of SARS-CoV-2 infection and increasing stringency in governmental measures. Gender, age, country, deprivation, inequalities, risk of bias, and study design did not modify these associations. LIMITATIONS: The certainty of the evidence was low because of the high risk of bias in included studies and the large amount of heterogeneity. Stringency measures and surges in cases were strongly correlated and changed over time. The observed associations should not be interpreted as causal relationships. CONCLUSION: Although an initial increase in average symptoms of depression and anxiety and an association between higher numbers of reported cases and more stringent measures were found, changes in mental health symptoms varied substantially across studies after the first 2 months of the pandemic. This suggests that different populations responded differently to the psychological stress generated by the pandemic and its containment measures. PRIMARY FUNDING SOURCE: Swiss National Science Foundation. (PROSPERO: CRD42020180049).

Occurrence and transmission potential of asymptomatic and presymptomatic SARS-CoV-2 infections: Update of a living systematic review and meta-analysis.

BACKGROUND: Debate about the level of asymptomatic Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) infection continues. The amount of evidence is increasing and study designs have changed over time. We updated a living systematic review to address 3 questions: (1) Among people who become infected with SARS-CoV-2, what proportion does not experience symptoms at all during their infection? (2) What is the infectiousness of asymptomatic and presymptomatic, compared with symptomatic, SARS-CoV-2 infection? (3) What proportion of SARS-CoV-2 transmission in a population is accounted for by people who are asymptomatic or presymptomatic? METHODS AND FINDINGS: The protocol was first published on 1 April 2020 and last updated on 18 June 2021. We searched PubMed, Embase, bioRxiv, and medRxiv, aggregated in a database of SARS-CoV-2 literature, most recently on 6 July 2021. Studies of people with PCR-diagnosed SARS-CoV-2, which documented symptom status at the beginning and end of follow-up, or mathematical modelling studies were included. Studies restricted to people already diagnosed, of single individuals or families, or without sufficient follow-up were excluded. One reviewer extracted data and a second verified the extraction, with disagreement resolved by discussion or a third reviewer. Risk of bias in empirical studies was assessed with a bespoke checklist and modelling studies with a published checklist. All data syntheses were done using random effects models. Review question (1): We included 130 studies. Heterogeneity was high so we did not estimate a mean proportion of asymptomatic infections overall (interquartile range (IQR) 14% to 50%, prediction interval 2% to 90%), or in 84 studies based on screening of defined populations (IQR 20% to 65%, prediction interval 4% to 94%). In 46 studies based on contact or outbreak investigations, the summary proportion asymptomatic was 19% (95% confidence interval (CI) 15% to 25%, prediction interval 2% to 70%). (2) The secondary attack rate in contacts of people with asymptomatic infection compared with symptomatic infection was 0.32 (95% CI 0.16 to 0.64, prediction interval 0.11 to 0.95, 8 studies). (3) In 13 modelling studies fit to data, the proportion of all SARS-CoV-2 transmission from presymptomatic individuals was higher than from asymptomatic individuals. Limitations of the evidence include high heterogeneity and high risks of selection and information bias in studies that were not designed to measure persistently asymptomatic infection, and limited information about variants of concern or in people who have been vaccinated. CONCLUSIONS: Based on studies published up to July 2021, most SARS-CoV-2 infections were not persistently asymptomatic, and asymptomatic infections were less infectious than symptomatic infections. Summary estimates from meta-analysis may be misleading when variability between studies is extreme and prediction intervals should be presented. Future studies should determine the asymptomatic proportion of SARS-CoV-2 infections caused by variants of concern and in people with immunity following vaccination or previous infection. Without prospective longitudinal studies with methods that minimise selection and measurement biases, further updates with the study types included in this living systematic review are unlikely to be able to provide a reliable summary estimate of the proportion of asymptomatic infections caused by SARS-CoV-2. REVIEW PROTOCOL: Open Science Framework (https://osf.io/9ewys/).

Outbreaks of publications about emerging infectious diseases: the case of SARS-CoV-2 and Zika virus.

BACKGROUND: Outbreaks of infectious diseases generate outbreaks of scientific evidence. In 2016 epidemics of Zika virus emerged, and in 2020, a novel severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) caused a pandemic of coronavirus disease 2019 (COVID-19). We compared patterns of scientific publications for the two infections to analyse the evolution of the evidence. METHODS: We annotated publications on Zika virus and SARS-CoV-2 that we collected using living evidence databases according to study design. We used descriptive statistics to categorise and compare study designs over time. RESULTS: We found 2286 publications about Zika virus in 2016 and 21,990 about SARS-CoV-2 up to 24 May 2020, of which we analysed a random sample of 5294 (24%). For both infections, there were more epidemiological than laboratory science studies. Amongst epidemiological studies for both infections, case reports, case series and cross-sectional studies emerged first, cohort and case-control studies were published later. Trials were the last to emerge. The number of preprints was much higher for SARS-CoV-2 than for Zika virus. CONCLUSIONS: Similarities in the overall pattern of publications might be generalizable, whereas differences are compatible with differences in the characteristics of a disease. Understanding how evidence accumulates during disease outbreaks helps us understand which types of public health questions we can answer and when.

Occurrence and transmission potential of asymptomatic and presymptomatic SARS-CoV-2 infections: A living systematic review and meta-analysis.

BACKGROUND: There is disagreement about the level of asymptomatic severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection. We conducted a living systematic review and meta-analysis to address three questions: (1) Amongst people who become infected with SARS-CoV-2, what proportion does not experience symptoms at all during their infection? (2) Amongst people with SARS-CoV-2 infection who are asymptomatic when diagnosed, what proportion will develop symptoms later? (3) What proportion of SARS-CoV-2 transmission is accounted for by people who are either asymptomatic throughout infection or presymptomatic? METHODS AND FINDINGS: We searched PubMed, Embase, bioRxiv, and medRxiv using a database of SARS-CoV-2 literature that is updated daily, on 25 March 2020, 20 April 2020, and 10 June 2020. Studies of people with SARS-CoV-2 diagnosed by reverse transcriptase PCR (RT-PCR) that documented follow-up and symptom status at the beginning and end of follow-up or modelling studies were included. One reviewer extracted data and a second verified the extraction, with disagreement resolved by discussion or a third reviewer. Risk of bias in empirical studies was assessed with an adapted checklist for case series, and the relevance and credibility of modelling studies were assessed using a published checklist. We included a total of 94 studies. The overall estimate of the proportion of people who become infected with SARS-CoV-2 and remain asymptomatic throughout infection was 20% (95% confidence interval [CI] 17-25) with a prediction interval of 3%-67% in 79 studies that addressed this review question. There was some evidence that biases in the selection of participants influence the estimate. In seven studies of defined populations screened for SARS-CoV-2 and then followed, 31% (95% CI 26%-37%, prediction interval 24%-38%) remained asymptomatic. The proportion of people that is presymptomatic could not be summarised, owing to heterogeneity. The secondary attack rate was lower in contacts of people with asymptomatic infection than those with symptomatic infection (relative risk 0.35, 95% CI 0.10-1.27). Modelling studies fit to data found a higher proportion of all SARS-CoV-2 infections resulting from transmission from presymptomatic individuals than from asymptomatic individuals. Limitations of the review include that most included studies were not designed to estimate the proportion of asymptomatic SARS-CoV-2 infections and were at risk of selection biases; we did not consider the possible impact of false negative RT-PCR results, which would underestimate the proportion of asymptomatic infections; and the database does not include all sources. CONCLUSIONS: The findings of this living systematic review suggest that most people who become infected with SARS-CoV-2 will not remain asymptomatic throughout the course of the infection. The contribution of presymptomatic and asymptomatic infections to overall SARS-CoV-2 transmission means that combination prevention measures, with enhanced hand hygiene, masks, testing tracing, and isolation strategies and social distancing, will continue to be needed.